A numerically efficient implementation of the expectation maximization algorithm for state space models

Empirical time series are subject to observational noise. Naïve approaches that estimate parameters in stochastic models for such time series are likely to fail due to the error-in-variables challenge. State space models (SSM) explicitly include observational noise. Applying the expectation maximization (EM) algorithm together with the Kalman filter constitute a robust iterative procedure to estimate model parameters in the SSM as well as an approach to denoise the signal. The EM algorithm provides maximum likelihood parameter estimates at convergence. The drawback of this approach is its high computational demand. Here, we present an optimized implementation and demonstrate its superior performance to naïve algorithms or implementations. Empirical signals are often obscured by a significant amount of observational noise. Observational noise is assumed to be white and Gaussian distributed, which is a justified assumption given the central limit theorem; it is further often assumed to enter the measurements as an additive effect. In contrast to dynamic noise, which adds to the dynamics of the process, observational noise is not part of the dynamics. In stochastic models both types of noise can be accounted for using the state space model (SSM). The SSM consists of an equation describing the dynamics of a process, and an observation equation, mod-eling the observation function and observational noise. For linear stochastic models, vector autoregressive (VAR½pŠ) processes are often used as a model for the dynamics. A VAR½pŠ is a versatile model which turns out to be powerful in estimating spectral characteristics, interaction structures, or network topologies. For example, the partial directed coherence [2,13] in the frequency domain and the directed partial correlation [4,7] in time domain, both measures for Granger causality, are based on parameters of vector autoregressive processes. Accurate parameter estimates are vital to get reliable results using such measures. Naïve estimators for parameters in the autoregressive model neglect observational noise. This leads to strongly biased estimates. The expectation maximization (EM) algorithm [3] provides an iterative maximum likelihood estimator [1] for